1. Field of the Invention
The invention relates to devices, methods, and systems for improving fault tolerance in computing systems. Specifically, the invention relates to devices, methods, and systems for managing virtual memory in fault tolerant multi-processor systems.
2. Description of the Related Art
Modem computer systems employ an operating system to manage system resources and to provide a foundation for application programs running on a computer system. The operating system provides a base for writing and running application programs, thereby freeing programmers from the details of computer system hardware. In addition, the operating system manages processes, memory, file systems, I/O systems, and the like.
In most operating systems, a process generally refers to a running program having inputs, outputs, state information, and a code image containing program codes. The state information may include the current values of the program counter, the registers, and the variables of an executing program. In many systems, each process consists of one or more threads that may be executed concurrently with each other. Processes and threads are well known in the art.
Running a process generally requires executing a thread that accesses the program codes and state information associated with the process. The program codes and state information are typically referenced by relative locations within a virtual address space associated with the process. Accessing the virtual address space typically involves utilizing memory management operations provided by the operating system and supporting hardware.
In particular, the operating system often includes a virtual memory system that maps virtual addresses associated with a thread from a large virtual address space to an actual physical address within a physical memory such as an array of RAM chips. In order to support virtual addressing, memory systems are often configured with multiple types or classes of memory. Some memory classes may be optimized for performance while other classes may be optimized for high capacity, low cost, removability, non-volatility or the like.
Virtual memory systems have been developed to facilitate sharing of the system memory by storing instructions and data in the several memory classes while giving the appearance that all elements are stored in the system memory. A virtual memory manager typically manages memory allocation, memory sharing, memory mapping and the like. A virtual memory address space is usually organized into memory address segments called pages, the page size often being determined by hardware considerations.
FIG. 1 is a block diagram illustrating a typical virtual memory system 100. The depicted virtual memory system 100 includes at least one process 110, a plurality of memory pages 114, a memory manager 120, a file system 130, a page table 140, a page map 150, at least one CPU 160, a storage controller 170, one or more storage devices 172 such as disk drives, a storage cache 180, a system memory 190, a CPU memory bus 122, a system data bus 124, and a system memory bus 126. The depicted virtual memory system 100 exemplifies many aspects of currently available virtual memory systems.
The process 110 has a thread (not shown) associated with a virtual address space (not shown), the virtual address space being organized into pages 114. The memory manager 120 allocates at least one page 114 in system memory 190 to the process 110. The memory manager constructs entries in a multiplicity of data structures such as the page table 140 that may contain entries such as a page usage log, a page lock status, a set of page sharing attributes, a page owner list and the like. The memory manager 120 may also construct entries in the page map 150 such as a virtual page address and a corresponding physical address of a page in the system memory 190.
The CPU 160 requests the contents of a virtual memory address by asserting the virtual memory address on the CPU memory bus 122. The memory manager 120 translates the virtual memory address to a physical memory address by mapping the entries in the page map 150. If the page containing the requested memory address is in physical memory, the memory manager 120 asserts the physical memory address on the system memory bus 126 and the corresponding data is asserted by the system memory 190 for use by the CPU 160. If the page containing the requested memory address is not in physical memory, the memory manager 120 typically generates a page fault. Page mapping and page faults are well known in the art.
In the event of a page fault, the memory manager 120 may subsequently request that the file system 130 access a page in the storage device 172 or the storage cache memory 180 that contains the requested memory address and copy the page to the system memory 190. The file system 130 typically issues a command to the storage controller 170 to retrieve the page containing the requested memory address. The storage controller 170 may determine if the page containing the requested memory address is in the storage cache memory 160, and if the page containing the requested memory is present, the storage controller 170 may copy the page to the system memory 190 by means of the system data bus 124.
In the event the page containing the requested memory address is not present in the storage cache memory 180, the storage controller 170 typically issues at least one read command to the storage device 172. The storage device 172 subsequently retrieves a set of memory blocks containing the page, and the storage controller 170 copies the page to the system memory 190. Subsequently, the memory manager 120 updates data structures associated with the newly copied page and asserts the physical memory address on the system memory bus 126. Significant performance degradation occurs whenever a page fault requires that a page be retrieved from the storage device 172.
In modern computer systems, operating systems generally allow multiple threads to execute virtually simultaneously in a virtual address space. For example, multiple threads on multiple CPUs could simultaneously perform page faults. Multiple threads may also execute a system call to map a file from a secondary storage device into the virtual address space. However, when multiple threads are attempting to access the same region in a virtual address space, a problem of contention arises. For example, if two threads are allowed to operate on the same virtual page in a region, the data may not be synchronized or updated properly. To address the contention problem, conventional techniques have used a “lock” to synchronize access by providing exclusive access to a thread such that other threads are not allowed to change the data accessed by the thread. In this manner, the lock ensures mutual exclusion of multiple threads for updates.
Data processing systems are often configured to improve fault tolerance by employing redundant elements. Several architectures are available that provide continuity of service with single point failures Some systems provide fault tolerant storage systems by using two or more storage processor nodes to control an array of redundant disk memories. If one storage processor node fails, another of the storage processor nodes can provide uninterrupted access to data stored on the array of redundant disk memories. Some systems use the multiple redundant storage processor nodes in a multiple parallel execution mode, each of the redundant storage processor nodes mirroring the operations of the other redundant storage processor nodes. Other systems typically use the two or more redundant storage processor nodes in a multiple active mode, wherein the two or more storage processor nodes execute concurrent unrelated threads. The multiple active mode results in better system performance in the case where the operation of the system requires significant disk memory activity.
Systems that use the two or more redundant processor nodes in multiple active (i.e. load sharing) mode may utilize virtual memory management techniques. Transparent fault recovery methods in systems that use processor nodes in multiple active mode generally require that data maps and state information for every active process be recovered. Maintenance of data and state information for all process threads typically requires that the system memory of each active processor node maintain synchronization with all other partner storage processor node memories.
What is needed are devices, methods and systems to maintain synchronization between the system memories of two or more processor nodes executing multiple disjoint threads in a virtual memory environment. The memory managers associated with the processor nodes need to manage memory allocation, memory sharing and memory mapping functions while simultaneously maintaining synchronization between system memories of all partner processing nodes. Such devices, methods and systems would provide the advantage of transparent system recovery in the event of a processor node failure in a system using multiple active processor nodes in a virtual memory environment.